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Reforming Testing and Accountability 
Essential Principles for Student Success in California 


California is in the midst of a fundamental remaking of its assessment 
and accountability system that has been in place for 15 years. The state 
has the opportunity to play a leadership role in developing a multi-faceted 
assessment and accountability system that goes beyond a narrow focus on 
test scores and that could be a model for the nation. 


The transition offers California the potential to develop a more effective system that will 
not only assess students in more meaningful ways across more parts of the school curricu- 
lum, but also will provide educators with meaningful information about their students to help 
drive academic improvement. 

The changes are being driven by several significant reforms, including: 

m@ The Common Core State Standards, adopted by the State Board of Education in August 2010, 

and the Next Generation Science Standards adopted in September 2013; 

M@ The dramatic reform of California’s school finance system, signed into law by Gov. Jerry 

Brown in July 2013; 

lM The passage of Assembly Bill 484 in September 2013— introducing a new statewide assess- 


ment system to replace the Standardized Testing and Reporting (STAR) program in place 
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In addition, Gov. Brown has opened up a debate in California—likely the only such debate 
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Richard Colvin. by any state in the nation—by questioning the overwhelming emphasis on testing in schools 
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The purpose of this EdSource report is threefold: to clarify the timeline for the revision 


must be incorporated into the new system for it to be most effective; and to make recommen- 


dations to ensure that whatever new system is developed happens in a timely way, and is most 


Because both assessment and accountability—how student academic abilities are assessed 


and how schools and districts are held accountable for student performance and their use 
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of taxpayer funds—are intimately related to each other, we examine both in 
this report. It is based on a nine-month review of the literature on testing and 
accountability, an examination of emerging models in other states, the tracking 


of key legislation, and interviews conducted with numerous education experts. 


Essential Principles of a Revised Testing 

and Accountability System 

EdSource has identified eight essential principles that should form the founda- 
tion of the state’s evolving testing and accountability systems. Many already are 
integrated into the reforms that are underway. For the new system to be most 


effective, reforms must move the state in the following directions: 


1. From a system with an excessive focus on standardized test scores to one 
that incorporates multiple measures that assess “deeper learning” and 


readiness for college and career success. 


2. From a system based solely on top-down accountability imposed by Sac- 
ramento and Washington, D.C., to one that uses local assessments and 


accountability. 


3. From a system that depends on tests whose results are issued once a year— 
and typically have no impact on how individual students are taught—to 
assessments that provide more immediate feedback in ways that help chil- 


dren learn and teachers teach more effectively. 


4. From a system based mainly on external rewards and punishments to one 
that incorporates intrinsic incentives that motivate change among indi- 
vidual students, teachers and schools, along with the resources they need 


to succeed. 


5. From a system that is focused mainly on getting children to perform at a 
“proficient” level to one that measures growth from year to year, motivates 
all children to do better, and encourages both students and schools to 


make progress at whatever level they are currently succeeding. 


6. From a system that focuses disproportionately on math and English language 
arts—often at the expense of other aspects of the school curriculum—to a 
more balanced curriculum that incorporates other key subject areas, 


especially science. 


7. From an overly complex system that is hard for ordinary Californians to 
understand to one that is more transparent and offers a multidimensional 


portrait of how students and schools are doing in clearer language. 


8. From an assessment system that uses technology mainly to report results to 
schools and the public to one that uses technology more innovatively to pro- 
vide more immediate feedback to teachers and students, and that tracks stu- 


dents’ progress through the 12th grade and into college and the workplace. 
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Recommendations 

1. Publish a clear timeline: To ensure that all major stakeholders are informed 
about the complex process for reforming the testing and accountability system, 
a clear timeline for all key milestones should be published by the California 
Department of Education in concert with the State Board of Education, and dis- 
tributed widely to policymakers, parent and community organizations, school 
board members, and other education stakeholders. 

2. Establish broader oversight: There is a danger that the various elements of 
the new testing and accountability system will evolve in an uncoordinated fash- 
ion, and in potentially conflicting ways. The State Board of Education should 
consider appointing an individual or group to oversee and coordinate the pro- 
cess with the Department of Education and other key constituencies to ensure 
that the necessary steps occur in a timely fashion, while still meeting the needs 
of students, schools and the state in general. 

3. Promote dialog on the role of testing: The State Board of Education 
should initiate a statewide dialog about the role of testing in the state’s emerging 
accountability system and the best way to evaluate a school’s performance. This 
will be especially important as California moves away from its reliance on the 
Academic Performance Index as the primary way to evaluate school perform- 
ance to one incorporating multiple measures as spelled out in school finance 
legislation and the Local Control and Accountability Plan that it mandates. 

4. Engage in a deliberative process: Policymakers must proceed with 
deliberation to put into place a new testing and accountability system—avoid- 
ing doing so in an overly hasty fashion that yields unintended consequences. 
Because most key pieces of California’s accountability system only need to be in 
place in 2015-16, policymakers have time to be discerning in crafting a new and 
coordinated strategy that builds on the best approaches being developed within 
California and in other states. 

5. Develop new science assessments: Development of new science assess- 
ments is lagging behind those created for English language arts and math. Edu- 
cators concerned about developing meaningful assessments in science should 
be encouraged to participate in stakeholder discussions that will be launched 
in the next several months. Key stakeholders in business, academia and public 
policy have the opportunity to play a greater leadership role at a crucial time 
to not only ensure that frameworks, curricula and assessments tied to the Next 
Generation Science Standards are developed in a timely way, but also to elevate 
the status of science within the school curriculum. 

6. Conduct an ongoing review: After the key parts of the system are in place, 
a mechanism for continual review of the reforms must be instituted to ensure 


that they are meeting the needs of educators, and that they are providing 
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educators with meaningful information about their students that contributes to 
greater academic success. In addition, parents and other community stakehold- 
ers should have a way to be involved in the statewide review process, paralleling 
their involvement at a local level. This will be especially necessary because of the 
system’s reliance on technology, which will inevitably undergo major changes as 
the reforms are being implemented. 

7. Provide adequate resources: While students and schools must be held 
accountable for achieving results, the state must provide the resources necessary 
for them to do so. California schools in general remain underfunded compared 
to most other states. As a result, there needs to be a mechanism to monitor 
the resources and funding available to districts and to make recommendations 
to the Legislature to ensure that schools have what they need to successfully 


implement the new testing and accountability system described in this report. 


Timing 

Because California’s emerging testing and accountability system is driven by at 
least three pieces of legislation approved at different times, and in an uncoordi- 
nated fashion, the timing for implementation is multilayered and complex. 

A major milestone will be the phase-out of the California Standards Tests 
and the administration of Smarter Balanced field tests in spring 2014. The full 
Smarter Balanced assessments will be administered for the first time in the 
spring of 2015. In March 2014, the State Board of Education suspended the Aca- 
demic Performance Index (API) until 2015-16, while the state makes the transi- 
tion to an API that reflects the Common Core-based tests as well as revisions 
to the high school API as a result of Senate Bill 1458. In 2015-16, the revised API 
will be implemented, along with new requirements for assessments of school 
performance in the eight “priority areas” spelled out in the school funding 
reform legislation signed by Gov. Brown in July 2013. 

Full implementation of the new system will be a long-term process, as dif- 
ferent pieces of the system will be rolled out at different times during the next 
several years, including new curricula and assessments in other subject areas, 


such as science. A full timeline can be found on pages 5 and 6. 
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ASSESSMENT AND ACCOUNTABILITY TIMELINE IN CALIFORNIA 


2010-11 


August 2010. The State Board of Education adopts the Common Core State Standards in math and English language arts. 


2011-12 


January 2012. Gov. Jerry Brown first proposes to reform the school financing system, including a “weighted student formula” targeting funds 
at low-income students, English learners and foster children. 


2012-13 


September 2012. Senate Bill 1458, authored by Senate President pro Tem Darrell Steinberg and signed into law by Gov. Brown, mandates, by 
the 2015-16 school year, the expansion of the Academic Performance Index at the high school level to include college- and career-readiness and 
local accountability measures. 


January 2013. Gov. Brown in his State of the State speech presents a revised proposal for reforming the school finance system, including the Local 
Control Funding Formula and an emphasis on local decision-making. 


State Superintendent of Public Instruction Tom Torlakson sends a report to Gov. Brown and the Legislature, Recommendations for Transitioning 
California to a New Assessment System. 


Modified versions of Common Core State Standards in mathematics and English language arts are adopted by the State Board of Education as 
recommended by the Academic Content Standards Commission. 


July 2013. Assembly Bill 97, signed by Gov. Brown, reforms the school finance system through a Local Control Funding Formula and calls for 
assessing schools in eight “priority areas” that go far beyond test scores. 


August 2013. Eight California districts that are part of the California Office of Education Reform (CORE) consortium, serving more than 1 
million students, receive a one-year waiver from many No Child Left Behind (NCLB) provisions. The waiver plan includes a new School Quality 
Performance Index, to be implemented in the 2014-15 school year. 


2013-14 


September 2013. The Common Core State Standards are implemented in California schools to varying degrees. 
The State Board of Education adopts the Next Generation Science Standards. 


October 2013. Assembly Bill 484, signed by Gov. Brown, adopts a new statewide assessment system, the California Assessment of Student Performance 
and Progress (CAASPP), intended to replace the Standardized Testing and Reporting (STAR) program. It requires Smarter Balanced field tests for Common 
Core standards in math and English language arts for grades 3 through 8 and 11 in place of California Standards Tests in spring 2014. 


November 2013. The State Board adopts a new math curriculum framework to provide guidance to teachers and administrators in implementing 
the Common Core standards. 


iz 


January 2014. The State Board adopts regulations for district use of “supplemental” and “concentration” grants based on enrollments of 


high-needs students as prescribed by the Local Control Funding Formula. 


The California Assessment of Student Performance and Progress (CAASPP) program, initially including only Smarter Balanced assessments, 
Officially replaces the STAR program. 


The State Board adopts a template for local districts for the Local Control and Accountability Plan. 
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March 2014. The State Board of Education suspends the Academic Performance Index for two years. 
May 2014. The State Board is expected to adopt a new curriculum framework for English language arts based on the Common Core. 


March-June 2014. Smarter Balanced field tests will be administered in 3rd through 8th grades and 11th grade in math and English language 
arts. Some 9th and 10th graders will also take the field tests. No results will be published. 


California Standards Tests and California Modified Assessments in science will be administered in grades 5, 8 and 10, as well as the California 
Alternate Performance Assessment (CAPA) in grades 2 through 11 for students with substantial cognitive disabilities. 


California Standards Tests, including those in history and social sciences, and tests in most high school math and science courses (in biology, 
chemistry, earth sciences and physics) will be eliminated. 


Spring—-December 2014. Groups of subject matter experts will draw up frameworks for the Next Generation Science Standards. 


July 1, 2014. Local education agencies (school districts, county offices, charter schools) have to adopt a three-year Local Control and 
Accountability Plan by this date. 


July 2014. The State Board of Education will adopt an English language arts framework to provide guidance to teachers and administrators in 
implementing the Common Core standards and an implementation plan for the Next Generation Science Standards. 


2014-15 


September 2014. Common Core State Standards will be implemented to varying degrees in schools across the state. 


March-June 2015. Computer-adaptive Smarter Balanced assessments on Common Core standards in English language arts and math will 
be administered in 3rd through 8th and 11th grades, and the results will be published for the first time. 


May 2015. The Public Schools Accountability Act Advisory Committee will make its final recommendations to the State Board for reform of 
the Academic Performance Index. 


July or September 2015. The State Board will approve the final revisions of the Academic Performance Index, based on recommendations of the 
Public Schools Accountability Act Advisory Committee. 


2015-16 


2015-16 School Year. The Academic Performance Index (API) will resume, reflecting results on the Smarter Balanced assessments as well as 
revisions to the high school API as a result of Senate Bill 1458. 


Oct. 1, 2015. The State Board will create guidelines or “rubrics” for determining how schools are doing in the eight “priority areas” identified in 
the Local Control Funding Formula legislation. 


March 2016. As prescribed by Assembly Bill 484, the State Superintendent of Public Instruction will provide a plan for additional student 
assessments, including history/social science, technology, and visual and performing arts. The Superintendent may also recommend additional 
assessments in English language arts, math and science, as well as other assessment modalities (e.g., reducing frequency of tests or doing matrix 
sampling so that not all students take all the annual tests). 


2016-17 


2016-17 School Year. The state will implement assessments for students whose primary language is one other than English. 


Districts begin full implementation of the Next Generation Science Standards. 


6 @ REFORMING TESTING AND ACCOUNTABILITY HM MARCH 2014 © CopyRiGHT 2014 BY EpSourcE, INc. 


EDSOURCE REPORT 


“The laws that are in fashion 
demand tightly constrained 
curricula and reams of 
accountability data. All 

the better if it requires 
quiz-bits of information, 
regurgitated at regular 
intervals and stored in vast 
computers. Performance 
metrics, of course, are 
invoked like talismans. 
Distant authorities crack 
the whip, demanding 
quantitative measures 

and a stark, single number 
to encapsulate the precise 
achievement level of 

every child.” 


— CALIFORNIA GOV. JERRY BROWN, 
2013 STATE OF THE STATE SPEECH 
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INTRODUCTION 


California is at a pivotal moment in reforming the testing and accountability 
system used to assess how well its more than 6 million public school stu- 
dents, and the schools that they attend, are doing. 

The transition presents a moment of enormous opportunity to develop 
a more effective system that not only assesses students in more meaningful 
ways, across more of the school curriculum, but also helps drive academic 
improvement. 

Much heralded state and federal reforms during the past 14 years have 
been narrowly focused on how students perform on standardized tests, along 
with consequences—often negative ones—for schools, teachers and princi- 
pals if students didn’t meet expectations set by Sacramento and Washington. 

Until now, the overwhelming emphasis at both the state and federal level 
has been on reading and math at the expense of other key parts of the school 
curriculum, including science, social studies and the arts, and with little 
regard to the “deeper learning” skills that students need to be successful in 
college and careers. 

Voluminous research that shows the high correlation between test scores 
and the income level of a student’s family has not guided past reforms. Also 
absent has been consideration of a range of non-school factors that have a 
direct impact on school outcomes, such as stresses students face in their 
homes and neighborhoods, their level of preparedness for school when enter- 
ing kindergarten, and undiagnosed or untreated health problems. Instead, 
the overwhelming emphasis has been on holding schools—and teachers— 
entirely responsible for how their students perform. 

Currently, almost every dimension of the testing and accountability 
system is under review, and California has an opportunity to emerge as a 
national leader in developing a new system that could be in place for another 


generation. 


New circumstances provide opportunities for reform 

Each of the following circumstances helped create the current opportunity. 

1. There is a deep sentiment among Republicans and Democrats, as well 
as educators and policymakers, that the federal No Child Left Behind 
(NCLB) law has not achieved its intended purpose. Despite this broad 
agreement, Congress has not managed to replace it, creating an opening 


for California to shape what comes next. 


2. California’s reform of its school funding system will also have a dramatic 
impact on what schools and districts will be held accountable for. School 


districts will be evaluated on how well they do in eight “priority areas” 
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NOTES 
1 ESEA Flexibility Waiver Application. 


2 “State Board adopts national K-12 Next Gen- 
eration Science Standards,” EdSource, Sept. 5, 
2013. 


3 Kirst, Michael, “The Common Core Meets State 
Policy: This Changes Almost Everything,” Policy 
Analysis for California Education, March 2013. 


that must be outlined in a “Local Control and Accountability Plan.” These 
include standard measures of student achievement—but go far beyond 
them to include how effectively schools involve parents and other novel 
dimensions, such as whether schools provide a positive “school climate” and 


are “engaging” students. 


The thrust of the new accountability system is to provide struggling schools 
with support services rather than impose punitive sanctions. An entirely 
new state agency, the California Collaborative for Educational Excellence, 


with an initial budget of $15 million, is being set up precisely for this purpose. 


In spring 2014, students for the first time will take a field test in English 
language arts and math aligned with the Common Core State Standards, 
replacing most multiple-choice tests that students in grades 2 through 11 
have been taking since the Legislature established the Standardized Testing 
and Reporting (STAR) program in the spring of 1998. 


The state already has set up a loosely defined California Assessment of Stu- 
dent Performance and Progress program (CAASPP), initially including 


only Smarter Balanced assessments, to officially replace the STAR program. 


A consortium of eight California school districts—the only such alliance in 
the nation—received a waiver from some of the most onerous provisions of 
the NCLB law. These California Office to Reform Education (CORE) dis- 
tricts are establishing a much broader “School Quality Improvement Index” 
as an alternative to the Academic Performance Index. The School Quality 
Improvement index will include not only multiple academic measures, but 
also measures of school climate, social and emotional well being, and other 
“non-cognitive” factors.’ These and other reforms could provide guidance 


for implementation of alternative measures in other districts. 


Senate Bill 1458, authored by President pro Tem Darrell Steinberg, D-Sac- 
ramento, mandates broadening the Academic Performance Index to include 
measures of how well schools prepare students for college and careers. The 
process for revising the API is still underway, and is slated to be introduced 


in the 2015-16 school year. 


The same legislation also calls for making science a more important part of 
the accountability system in California. In addition, in September 2013, the 
State Board of Education adopted the Next Generation Science Standards 
(NGSS), though without a timeline for their implementation.’ 


Affecting all of the above are the Common Core State Standards, which 


State Board of Education President Michael Kirst predicts will “impact almost 


all key state education policies in fundamental ways.”? 
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4 Governor's Revision to Budget, May 2011. 


5 “Brown lashes out at regulators and testers, 
makes case for reform,” EdSource, Jan. 24, 2013. 


Also State of the State Address, Jan. 13, 2013. 


6 “Recommendations for Transitioning California to 
a Future Assessment System,” California Depart- 
ment of Education, Jan. 13, 2013. 


7 Comments at Policy Analysis for California 
Education Seminar, Sacramento, Oct. 3, 2013. 


More emphasis on local decision making 
These developments are all taking place at the same time that Gov. Jerry Brown 
is expressing deep skepticism about the emphasis on testing that has dominated 
the school reform agenda for more than a dozen years. As his administration 
outlined in the May revision to his 2011 budget: 
Testing takes huge amounts of time from classroom instruction. Data 
collection requirements are cumbersome and do not provide timely— 
and therefore usable—information back to schools. Teachers are forced 
to curb their own creativity and engagement with students as they focus 
on teaching to the test. State and federal administrators continue to cen- 
tralize teaching authority far from the classroom.* 

In addition, Gov. Brown has been pushing to drive more control to a local 
level, based on the principle of “subsidiarity” drawn from his years as a student 
in a Jesuit seminary. 

Subsidiarity is the idea that a central authority should only perform those 
tasks which cannot be performed at a more immediate or local level.‘ 
This principle has also been at the core of Gov. Brown’s reform of the state’s 


opaque school finance system that had been in place for decades. 


Fundamental shift underway 
Thus, on the testing side of the equation, California is in the midst of a funda- 
mental shift in the way it assesses students’ performance and, on the account- 
ability side, how it holds schools accountable for how students do on the tests. 
As State Superintendent of Public Instruction Tom Torlakson noted in his far- 
reaching January 2013 report: 

The time has come to remake our state’s assessment system. As we do, we 

must set our sights on a new, more ambitious goal—creating a system 

that fosters high quality teaching and learning in every classroom.® 

At the same time, Deb Sigman, Torlakson’s deputy superintendent of public 
instruction and the point person in the California Department of Education on 
testing and accountability, cautioned that “this is a very long-term project” that 
began in the 1980s with its emphasis on minimum competencies, continued into 
the 1990s with its focus on students being proficient, and is motivating the cur- 
rent reforms with their emphasis on college and career readiness. 
Referring to Assembly Bill 484, the key legislation driving the reforms, she said: 

This is a powerful and historical transformation. It (AB 484) recognizes 

that you can’t get there in a day. It recognizes that there really is a tran- 

sition period that needs to be reasonable, and deliberate, and that allows 

people the time and space to focus in on the instruction and delivery of 


college- and career-ready standards. 
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Key legislation that is shaping 
testing and accountability 


Senate Bill 1458. Legislation sponsored 
by Senate President pro Tempore Darrell 
Steinberg, D-Sacramento, and signed into 
law by Gov. Jerry Brown in September 2012, 
to broaden the Academic Performance Index 
to include college and career readiness and 
other measures that go beyond test scores. 
SB 1458 also allows for the establishing of 
“locally convened panels to visit schools, 
observe teachers, interview pupils and 
examine pupil work.” 


Assembly Bill 97. Budget bill signed by 
Gov. Brown in July 2013, which reforms 
the school financing system, requires each 
district to draw up a Local Control and 
Accountability Plan, and establishes eight 
“priority areas” on which schools will be 
assessed. AB 97 also calls for more local 
decision making and parent and community 
involvement. 


Assembly Bill 484. Signed by Gov. Brown in 
September 2013, it adopts a new statewide 
assessment system intended to replace the 
Standardized Testing and Reporting (STAR) 
program. AB 484 requires Smarter Balanced 
field tests for Common Core standards in 
math and English language arts in spring 
2014 in grades 3 through 8 and 11 in place 
of California Standards Tests. 
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Coordinating reforms will be challenging 

Despite the great opportunity for reform, this will also be an extremely chal- 

lenging period for educators in California, mainly because reform initiatives are 

occurring on multiple levels, each with its own timeline. 

Several pieces of legislation have gone into effect at different times, mandat- 
ing changes to different aspects of the assessment and accountability contin- 
uum, and creating some confusion as to how all the pieces of reform match up 
with each other. 
m™ Approved September 2012: Senate Bill 1458 is intended to reform the high 

school Academic Performance Index. 

m@ Approved July 2013: Assembly Bill 97, the Local Control Funding Formula, 
mandates a broader set of accountability measures—described as “priority 
areas — than just the Academic Performance Index. 

m@ Approved September 2013: Assembly Bill 484 institutes a new statewide 
assessment system intended to replace the Standardized Testing and Report- 
ing (STAR) program. 

Although the State Board of Education will have a key role—in tandem with 
the California Department of Education under the leadership of the state super- 
intendent of public instruction—there is no single body that is charged with 
coming up with a unified system of testing and accountability. 

Also involved in the process are national organizations, such as the Smarter 
Balanced Assessment Consortium, a partnership of 25 states established to cre- 


ate a new set of tests based on the Common Core standards. 


2015-16 will be a key year for implementation 
The key year for implementation of a new testing and accountability system in 
California is 2015-16. 

In that school year, Senate Bill 1458 mandates that a revised Academic Per- 
formance Index will be in place, at least on the high school level. That is also the 
year the State Board of Education will approve regulations for how schools will 
be measured on their ability to succeed in the eight “priority areas” outlined 
in the school finance legislation. By then, students will already have taken the 
Common Core Smarter Balanced assessments the previous spring in reading 
and math, so that by 2015-16 there will be a baseline to measure their progress. 

However, it will take several years longer before a complete accountability 
system will be fully in place. It is likely that new assessments in history and 
social science, as well as high school end-of-course tests in math and science, 
will still be under development, underscoring the magnitude of the task facing 
the state. 
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CURRENT ACCOUNTABILITY SYSTEM IS URGENTLY IN NEED OF REFORM 


For the past decade, California has operated under dual—and dueling— 
accountability systems. One is an accountability system established by 
the state through California’s Public Schools Accountability Act of 1999. 
The other is the accountability system imposed by the federal government 
through the federal No Child Left Behind Act, signed into law in January 
2002 by then-President George W. Bush. 


Both are in urgent need of reform for at least the five following reasons: 


1. Federal and state accountability systems have resulted in con- 
tradictory assessments of performance. 

The state and federal accountability systems have produced conflicting 
outcomes, sending confusing messages to educators, parents, students 
and the public at large as to whether their schools are succeeding or not. 
Depending on which system they consult, Californians might be told that 
the same school is succeeding or failing. In other words, some schools 
may have reached their growth target as set by the state on the Academic 
Performance Index, but not have achieved “adequate yearly progress” as 
defined by the No Child Left Behind law either for the entire school or for 
any one of numerous student “subgroups” in the school. 


An EdSource analysis of more than 7,000 California schools showed that in 
2012, 57 percent were succeeding on one of three measures of progress 
set by the state’s standards, but were simultaneously deemed not to have 
made “adequate yearly progress” under federal NCLB standards.® 


The confusion over whether schools are succeeding or failing is not a new 
development, but has characterized California's school accountability 
system ever since the introduction of the No Child Left Behind law in 2002. 


2. Even though student scores have improved substantially on the 
California Standardized Tests, especially at the elementary and 
middle school levels, both federal and state systems have failed 
to come even close to closing the achievement gap.° 

The accountability systems, introduced at enormous cost and imposing 

multiple requirements on educators, have not resulted in closing the 

achievement gap. 


While some schools have made more progress than others (as one would 
expect in a state with nearly 10,000 schools), on average the test score 
gap between white and Asian students on the one hand, and African 


NOTES 


American and Latino students on the other, remains far too high—between 
20 and 30 percentage points on the California Standards Tests. 


A similar pattern exists on the National Assessment of Educational 
Progress. Although some progress has been made since the early 1990s— 
and much less during the past decade—enormous gaps still exist. In 4th 
grade reading in 2013, NAEP noted that “black students had an average 
score that was 30 points lower than white students. This performance gap 
was not significantly different from that in 1992 (36 points).” Similarly, 
the performance gap between Hispanic and white students had also not 
changed significantly since then, according to NAEP.° 


3. The federal accountability system has been especially ineffec- 
tive in getting students and schools to meet the performance 
goals established more than a decade ago. But even under 
the state system, despite progress, many schools still have 
not reached the state target of an 800 score on the Academic 
Performance Index. 

Achieving the NCLB goals has been especially elusive, mainly because the 

law has set an impossible goal for California, as well as all other states, to 

achieve. Under the law, every student in every school is supposed to score 
at a “proficient” level on state English language arts and math tests by the 
end of the 2013-14 school year. 


On the most recent battery of state tests administered in the spring of 
2013, 56.4 percent of students in California scored at a proficient level or 
higher on California’s English language arts test, and 51.2 percent did so 
on math tests. On the positive side, these figures represent a significant 
increase in proficiency levels since 2003, when only 35 percent of 
students scored at a proficient level in both math and English. But they 
fall far short of the 100 percent proficiency prescribed by the NCLB law. 
State leaders also had expectations that during a 15-year period well 
more than half of its students would be performing at a proficient level.” 


4.A large number of schools have not turned around in response 
to the range of sanctions contained in both the state and 
federal accountability systems. 

In tandem with a substantial number of students failing to score at a 

proficient level on state tests, increasing numbers of schools have been 

labeled as essentially failing under NCLB. 


8 Under the state’s accountability system, schools are evaluated on whether the school 1) met a schoolwide growth target for improved test scores; 2) all of the school’s student 
subgroups met their growth targets; or 3) the school and all student subgroups met their growth targets. 


9 2013 STAR results. 
10 NAEP’s 2013 Report Card, Reporting Snapshot on California. 
11 STAR Program, Summary of 2013 Results. 
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At the beginning of the 2013-14 school year, 90 percent of California’s 
6,135 schools receiving federal Title 1 funds for low-income children 
have been labeled as being in need of “Program Improvement,” the 
effective equivalent of a failing school under the NCLB law. A school is 
deemed in need of Program Improvement if it fails to make “adequate 
yearly progress,” as measured by NCLB, two years in a row. It is likely 
that by the end of the current year, the vast majority of Title | schools will 
receive this failing designation. 


By contrast, schools did much better as measured by the state’s 
standards. API scores have generally risen, and an API score that once 
would have placed a school above the median for the state now would 
place it in the bottom rung of schools. 


Yet, large numbers of schools have also failed to reach the more modest 
growth targets set by the state’s accountability system. By 2012, after 
nearly 15 years of effort, only 53 percent of schools reached or exceeded 
the state’s goal of 800 on the API, underscoring how far the state still 
has to go. 


5. Both systems assess only a portion of the school curriculum. 
Both federal and state laws have measured only a portion of what a 
student is expected to learn while in school. Both systems are heavily 
skewed to math and English language arts, either minimizing or leaving 
out altogether what students are learning in other subjects. 


To determine whether a school is making “adequate yearly progress” as 
defined by the federal NCLB law, only test results in math and English 
language arts and proficiency rates on the California High School Exit 
Exam (which measures only English and math) are used.’ 


NOTES 


The state’s Academic Performance Index is based on a somewhat 
broader set of tests. Under the state’s system, in addition to math and 
English language arts, test scores in science in grades 5, 8 and 10, 
history-social science in grades 8 and 11 and, for grades 9-11, end-of- 
course, single-subject tests in science (biology, chemistry, earth science 
and physics) and world history are included in calculating API scores. 


But assessments of how students are doing in music, art, physical 
education and foreign language courses are absent from both federal 
and state systems, as are other indicators of a school’s effectiveness, 
such as attendance and suspension and graduation rates. Science 
assessments, while given in some grades, continue to take a back seat 
to those in math and English language arts. 


As Kevin Carey, director of education policy for the New America 
Foundation, has noted, “accountability systems that ignore much of 
what schools expect from students suffer from a fundamental lack of 
legitimacy.” 


At a minimum, Californians deserve a much clearer set of accountability 
standards that don’t conflict with each other, and which give them a 
sharper sense of the state’s return on its annual investment of tens of 
billions of dollars in its public schools. 


They also deserve an accountability system that measures all skills that 
children are acquiring in school, not just their answers on multiple- 
choice questions in math and English language arts. 


12 2012 Adequate Yearly Progress Reports Information Guide, California Department of Education. 


13 Ibid. 


14 See Carey, Kevin. “Some Assembly Required: Building a Better Accountability System in California,” Education Sector, April 2012. 
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“This is a great time 

to engage in a broader 
community conversation 
about what a high quality 
school looks like to... various 
constituency groups, and 
what indicators or measures 
can be collected to document 
that quality.” 

— CHUCK WEIS, FORMER SANTA 


CLARA COUNTY SUPERINTENDENT 
OF SCHOOLS 


NOTES 
15 Senate Bill 1458. 


16 Polikoff, Morgan and Andrew McEachin. Fixing 
the Academic Performance Index, PACE, January 
2013. 


17 PSAA minutes, June 25, 2013. 


Webcasts of the meetings on the PSAA Advisory 
Committee are also available on the Department 
of Education website. 


18 List of Advisory Committee members. 


ESSENTIAL PRINCIPLES FOR AN EFFECTIVE 
ASSESSMENT AND ACCOUNTABILITY SYSTEM 


EdSource has identified eight essential principles that should form the foun- 
dation of the state’s evolving assessment and accountability system. Many are 
already integrated into the reforms that are underway. For the new system to be 


most effective, reforms must move the state in the following directions. 


ESSENTIAL PRINCIPLE #1: From a system with an excessive focus on 
standardized test scores to one that incorporates multiple measures 
that assess “deeper learning” and readiness for college and career 
SuCcCeSS. 

Remaking California’s testing and accountability system must begin with 
rethinking and reforming the Academic Performance Index, which has been 
the main measure, instituted by the state in 1999, to hold schools “accountable” 
for improving tests scores from one year to the next. 

In March 2014, the State Board of Education voted to suspend the API 
until 2015-16, while the state makes the transition to an API that reflects the 
new Common Core-based tests, as well as revisions to the high school API as a 
result of Senate Bill 1458.’ However, the suspension of the API does not address 
some fundamental concerns that have been raised about its composition and 
usefulness. 

The multiple flaws in the current API have been well documented. The clear- 
est exposition of why it is in need of reform can be found in a Policy Analysis for 
California Education paper titled Fixing the Academic Performance Index.’* Its 
authors outlined in compelling fashion eight problems with the API that “limit 
its potential utility and effectiveness.” These include the API’s narrow focus on 
test scores, its inability to track how well—or poorly—individual students are 
doing from year to year, and the fact that API scores correlate highly with pov- 
erty levels of students in a school or district. 

The 14-member Public Schools Accountability Act (PSAA) Advisory Com- 
mittee has been meeting regularly to come up with the best ways to implement 
SB 1458, including, for example, how to incorporate graduation rates in a school’s 
API.” The advisory committee consists of testing experts, officials in charge of 
testing in their districts, charter school representatives, and representatives of 
organizations such as the California Teachers Association and the California 
State PTA.” 

Even members of the PSAA advisory committee are questioning the API’s 
usefulness, especially in the context of Gov. Brown’s call for broader and more 
locally driven assessments. As a result, the index may end up as just a small por- 
tion of the state’s testing and accountability system, rather than maintaining the 


dominant role it has played in the past. 
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19 Interview of Ting Sung done by Richard Colvin 
for EdSource. 


Chuck Weis, a former Santa Clara County superintendent of schools who has 
been on the PSAA Advisory Committee since its inception in 1999, described 
the API as a “rudimentary measure” that may have outlived its usefulness. 

“T think it’s time to rethink the API, rather than continue to try to squeeze 
new, more robust information into an old system,” he told EdSource. 

Weis says what is needed is a “broader conversation” about the kind of 
accountability system the state needs. “We should not just tinker with the API 
and then slap it back out there again because it really does not do what we want 
it to do,” he said. 

Even incorporating measures of how prepared students are for college and 
careers—as called for in Senate Bill 1458—is presenting the PSAA committee 
with considerable challenges, said Ting Sung, the committee’s chair. She noted 
that critical thinking and creativity are essential dimensions of college and 
career readiness. “These are more difficult to measure,” she said, “but we know 
that these are skills that are needed for students to be more successful in life— 
no matter what job or career path they choose.”” 

But what could have the greatest impact on the role of the API are the eight 
priority areas mandated by the reforms of California’s school financing system 
(see figure below). Under this new system, the API is only one of seven meas- 
ures of student achievement on which a school’s effectiveness will be assessed, 
and only one of nearly two dozen criteria overall, most of which are unrelated to 


testing (such as parent involvement, student “engagement” and school climate). 


Toward broader measures of accountability 
Districts will be held accountable not just for test scores, but also for results in 
eight priority areas. 


i 
— ==) 
a 


EdSource, 2013 
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California is thus moving in the direction of broadening the number of meas- 
ures on which schools will be held accountable by parents, community mem- 
bers and other key stakeholders. 

However, the state will face significant challenges in implementing a broader 
accountability system. A review by the RAND Corporation of multiple meas- 
ures incorporated in accountability systems in 20 states concluded that little is 


known about their effectiveness. RAND researchers found: 


‘There is almost no published research about the technical quality of the 


measures, the theories of action that instigated their adoption, the utility 


of the measures for promoting improved decision making, or the effects 


of the measures on school practice or student outcomes.” 


They also cautioned that: 


Although the arguments in favor of expanded measures are strong, those 


responsible for setting policy or designing systems must proceed with 


caution and should be armed with an understanding of the various risks 


involved.” 
NOTES 


20 Schwartz, Heather, et al., Expanded Measures 
of School Performance, RAND Corporation, 2011. 


These risks include having so many measures that educators won't know 


where to focus their efforts, and that the system becomes so complex that it 


becomes harder for parents and others to get a sense of the quality of their 


21 Ibid. 


schools. California may also find that getting results on these multiple measures 


will cost more than it is willing—or able—to pay. 


Funding reforms emphasize eight priority areas for schools 


In their Local Control and Accountability Plans prescribed by California 
school funding reforms, school districts will be expected to focus on eight 
priority areas and set goals in each. The eight priority areas fall into three 
categories. 


Conditions of learning 

& Access to core services as measured by the extent to which students are 
taught by fully credentialed teachers, have standards-aligned textbooks 
and materials, and attend classes in safe and clean facilities. 


m /mplementation of the Common Core State Standards, the Next 
Generation Science Standards and other academic standards. 


= Access to a broad course of study and programs for high-needs and 
exceptional students. One measure will be enrollments in all courses 
required for admittance to California’s public universities. 


Student outcomes 
™ Student achievement as measured by performance on standardized 
tests, the Academic Performance Index, the proportion of students who 
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graduate “college and career ready,” the proportion of English learners 
who are reclassified as fluent in English, the share of high school students 
who pass Advanced Placement course exams with a score of at least a 3 
out of 5, and other measures. 


@ Student outcomes as measured by performance in other required areas of 
the school curriculum, such as physical education and the arts. Other forms 
of assessments, such as SAT or ACT college entrance examination scores, 
could also be included. 


Engagement 


mw Student engagement as measured by graduation and middle and high 
school dropout rates, chronic absenteeism and attendance. 


mw Parent involvement as measured by the extent to which parents participate 
in key school decisions. 


m School climate as measured by suspension and expulsion rates, and other 
measures chosen by local school districts. 
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In reality, most school leaders and teachers are under pressure to meet the expec- 


tations of multiple constituency groups. How and whether these expectations can 


be formally integrated into a unified accountability system is another question alto- 


gether. As Chuck Weis, the former Santa Clara County superintendent of schools, 


has urged: “This is a great time to engage in a broader community conversation 


about what a high quality school looks like to these various constituency groups, 


and what indicators or measures can be collected to document that quality.” 


Legislators call for college and career readiness measures in the Academic Performance Index 


California is on track to reform the main measure of school performance, the 
Academic Performance Index, or API. 


Senate Bill 1458 specifies that scores on standardized tests shall not comprise 
more than 60 percent of the high school Academic Performance Index. 


It instructs the state superintendent of public instruction to come up with a 
plan to include other measures of a school’s effectiveness in the API, such as 
indices of the college and career readiness of graduating students, the rates 
at which middle and high school students are promoted from one grade to 
the next, and high school graduation and drop-out rates. 


The process of reforming the API is being carried out primarily through the 
14-person Public School Accountability Act Advisory Committee, which meets 
quarterly. Every six months, the committee presents an interim report to the 
State Board of Education, with its final report due in May 2015. The State 
Board is slated to make a final decision on revisions of the API in July 2015. 


To assist the committee, the California Department of Education has 
contracted with David Conley from the University of Oregon to help define 


what indices should be used to assess whether students leave schools ready 
for college and careers. These could include the number of students in 
Advanced Placement classes, students’ SAT and ACT scores, and whether 
they are simultaneously enrolled in both high school and college courses 
(typically referred to as dual enrollment). 


The law also authorizes the state superintendent to come up with alternatives 
to the current system of giving schools a decile ranking of 1 to 10 based on 
their API score relative to all schools in the state—yet another indicator of the 
unhappiness with the current assessment systems. These rankings are highly 
correlated with the income levels of a school’s student population. 


Adding to the unhappiness with the API is that even if a school’s API score 
has increased substantially during the past dozen years, it can still get a 
lowly decile ranking compared to other higher-ranked schools whose scores 
have also risen. “By definition, no matter how well everyone does, 10 
percent of schools will always be in the bottom decile,” said Ed Haertel, a 
Stanford University education professor and a member of the Public Schools 
Accountability Act Advisory Committee. 
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The key to having more local 
assessments would be to 
gather information that could 
be useful and contribute 

to improved outcomes, 

and not become just a 
bureaucratic—and ultimately 
useless—exercise. 


ESSENTIAL PRINCIPLE #2: From a system skewed toward top-down 
accountability imposed by Sacramento and Washington to one that 
incorporates local accountability and assessments. 

Gov. Brown has introduced a major new dimension into the testing and 
accountability debate by insisting that local schools and districts be more 
involved in evaluating their own performance than they have been in the 
recent past. 

This marks a significant shift away from the testing and accountability sys- 
tems that have been based on top-down edicts in the form of legislation from 
Sacramento or Washington. The NCLB law, in particular, represents the most 
significant attempt by the federal government to require schools to deviate from 
the principle of local control of education on which the nation’s public school 


system was founded. 


In a memorable veto” of an earlier version of Senate Bill 1458, authored by 


Senate President pro Tem Darrell Steinberg, Gov. Brown wrote: 


A sign hung in Albert Einstein’s office read, “Not everything that counts 


can be counted, and not everything that can be counted counts.”... 


‘There are other ways to improve our schools—to indeed focus on qual- 


NOTES 


ity. What about a system that relies on locally convened panels to visit 


schools, observe teachers, interview students and examine student work? 


22 Gov. Jerry Brown’s veto message, Oct. 1, 2011. 


Such a system wouldn't produce an API number, but it could improve 


the quality of our schools. 


Common Core State Standards will reshape California’s testing and accountability systems 


California has joined 44 other states and the District of Columbia in 
adopting the Common Core State Standards in English language arts and 
mathematics.” The goal of these standards, developed under the aegis of 
the National Governors Association and the Council of Chief State School 
Officers, is to ensure that students will be ready for college-level courses or 
training programs for high-skill, high-wage careers by the time they graduate 
from high school. In particular, the standards place a strong emphasis on 
critical thinking skills and problem solving. 


The standards are voluntary. School districts are not required to implement 
them, though they are required to administer the Smarter Balanced 
assessments, which are based on the standards. School districts have been 
preparing to teach to those standards by providing training opportunities 


NOTES 


23 Frequently Asked Questions, Common Core State Standards Initiative website. 
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for their teachers, developing new curricula, and purchasing appropriate 
hardware, software and other materials. But teacher preparation activities 
have varied enormously from one district to another, raising concerns that in 
many schools and districts teachers have not received the training necessary 
to effectively implement the Common Core standards. 


One measure of the magnitude of the changes required is that California 
is giving school districts $1.25 billion over and above what they would 
have otherwise received to help them gear up for the new standards. All of 
that preparation must be done quickly because, beginning in spring 2014, 
students will take Smarter Balanced field tests based on the Common Core 
standards—and in the spring of 2015, they will take the full battery of tests 
for the first time. 
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NOTES 


24 Jerald, Craig, On Her Majesty’s School Inspec- 
tion Services, Education Sector, 2012. 


25 See a detailed description of Schoo! Quality 
Reviews from the Charlotte-Mecklenburg Schools. 


A year later, the notion of local review panels was incorporated into Senate 
Bill 1458, which gives the state superintendent of public instruction the author- 
ity to “develop and implement a program of school quality review,” including 
the locally convened panels Gov. Brown referred to in his earlier veto. 

The idea of locally convened panels is loosely based on the British system of 
“inspectorates,” which was the subject of a report produced by Education Sector 
titled On Her Majesty's School Inspection Service.** The report asserted that: 

Inspections offer a way to make much more nuanced judgments about 
school performance, provide richer information to parents and the pub- 
lic, offer better formative feedback to schools, inform much more tar- 
geted improvement and interventions for low-performing schools, and 
accelerate timelines for school improvement. 

The Education Sector report estimated that establishing an inspection ser- 
vice in California similar to the one that exists in the United Kingdom would 
require more than 800 inspectors, and would cost between $65 million and 
$130 million annually. Although these costs were far too high to consider during 
the budget crisis of the past several years, that may no longer be the case in light 
of the state’s return to financial health. 

The estimated costs could be significantly reduced if some of the inspectors 
were drawn from existing staff at county offices of education. Unlike in Cali- 
fornia, the United Kingdom does not have these intermediate education agen- 
cies that could play an important role in implementing an inspectorate-type 
program. 

Although no state or district in this nation operates a comprehensive system 
like the one in place in the United Kingdom, similar efforts in the United States 
and California could provide a model that the state could build upon. 

In 2006, New York City schools contracted with the English firm Cambridge 
Education to conduct two-day “School Quality Reviews” as part of its account- 
ability system.” Since 2008, Ohio has conducted what it calls School Improve- 
ment Diagnostic Reviews. Twenty-four state officials conduct two-day site 
visits of “underperforming” schools. They identify the schools’ strengths and 
weaknesses, and make a series of recommendations in the form of a “diagnostic 
report.” Closer to home, Oakland has developed its own indigenous version of 
Cambridge’s School Quality Review (see box on page 19). 

A little-known provision of the No Child Left Behind law—Section 1117— 
prescribes “local support teams” to assist struggling schools, which could also 


be helpful in building the local input and evaluations Gov. Brown has in mind. 
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NOTES 


26 Provision in State Education Code 52059, 
Article 4.2. 


27 Evaluation of the California Statewide System 
of School Support (S4): Final Report, Human Re- 
sources Research Organization, 2009. 


This NCLB mandate has received almost no public attention in California and 
in most other states. 

In response to this NCLB mandate—and the daunting task of trying to assist 
thousands of struggling schools after the passage of the NCLB law in 2002— 
California established a regional system of support based in county offices of 
education called a Statewide System of School Support, or $4. The goal was to 
build a district’s capacity “to support low-performing schools.””° 

These support teams have been virtually invisible in the statewide and 
national reform debates. But an in-depth final evaluation conducted by the 
Human Resources Research Corporation (HumRRO) for the California 
Department of Education found “evidence throughout the state that S4 is con- 
tributing positively to district and school needs.”” 

Thus, school districts would not have to start from scratch in carrying out 
more local assessments should California move in this direction. The key would 
be to gather information that would be useful and contribute to improved out- 


comes, and not become just a bureaucratic—and ultimately useless—exercise. 


Oakland Unified reviews schools with local panels 
Oakland Unified School District instituted a local evaluation program in 
2011-12 that could serve as a model for other districts. It was prompted by 
the Cambridge School Quality Review approach, but with some significant 
differences. 


In Oakland, reviews are carried out by “SQR” teams, consisting of specially 
trained principals and teachers from outside the schools. These reviews have 
so far been conducted in 52 schools. High school review teams sometimes 
also include high school students. The process begins with a “self reflection” 
report by the school to be reviewed, followed by an analysis of student 
outcome data. The SQR team then conducts a three-day site visit and holds 
a series of interviews, focus groups and meetings with students, families, 
community partners, teachers and administrators. 


Schools are rated on 25 different standards. But David Chambliss, who runs 
the program and has the title of “director of Continuous School Improvement,” 
said the process is about “continuous improvement, not a report card.” 
Rather than coming up with specific recommendations, the review teams 
identify the “strengths” and “challenges” of a particular school. 


Chambliss noted a big difference between Oakland's review teams and those of 
Cambridge Education, the inspiration for Oakland’s model. Oakland’s teams are 
indigenous to the district—not run by outside consultants. Cambridge’s teams 
typically involve district staff but, Chambliss said, “the leaders of the teams are 
Cambridge folks, and ultimately the authors are the Cambridge folks.” 
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This district-based approach has made a big difference in how the review 
teams are received by staff at local school sites. The goal is to engage 
schools in the process, rather than to set up an adversarial relationship with 
the review team. In addition, the district makes a conscious effort to involve 
parents in the process. Before the team arrives, they hold focus groups with 
parents. “We want to enlist them as partners to engage with us around our 
findings,” he said. 


Aunique aspect of the program is that the district has set up a parallel review 
process for its central office operations, which is informed by issues raised 
by principals and others during the school reviews. This takes into account 
the reality—however uncomfortable it may be to school administrators—that 
how a school functions is affected in crucial ways by the effectiveness or 
efficiency of the district’s central operations. According to Chambliss: 


We are building a central office quality-review process on top of the 
School Quality Review. Hopefully, we get a top-to-bottom alignment. 
When you start reviewing schools, you quickly start uncovering the 
challenges you have in central services. School personnel know that 
data won't get lost, it is no longer anecdotal, and you are collecting 
systematic evidence across schools about central service quality. 
Ultimately that becomes a call-out of where you have excellence and 
where you don't. 
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Surveying parents, students and teachers: A key tool for local accountability 


Getting input and observations of key local education constituencies— 
parents, students and school personnel—are an important dimension of 
local accountability. 


In fact, the 2013 funding reform legislation (Assembly Bill 97) lists surveys 
of pupils, parents and teachers on their “sense of safety and school 
connectedness” as one way to measure “school climate” in their district’s 
Local Control and Accountability Plan.” 


At least five school districts—New York City, Chicago, Washington, D.C., 
Los Angeles, San Jose and Oakland—conduct surveys of parents, students 
and staff on a significant scale and incorporate their views into their 
accountability reports. 


Only two states—Kentucky and Rhode Island—conduct surveys of all their 
schools. The TELL Kentucky survey is an annual 35- to 40-minute survey 
given to all teachers and principals. In 2013, some 43,000 teachers and 
principals responded—and 90 percent of Kentucky’s schools met the 50 
percent minimum response rate. 


Rhode Island’s SurveyWorks! initiative, conducted with help from WestEd, an 
education research organization, is more comprehensive. It surveys parents, 
teachers and students over a one-month period (in 2013 between Nov. 12 
and Dec. 13, for example). Reports based on the survey are then given to 
each school through a program it calls Infoworks!. 


The New York City School Survey, administered each year since 2007, is the 
largest survey in the nation except for the U.S. Census, and it is arguably 
even more ambitious. It polls 1.6 million parents, teachers and children in 
the sixth grade and up. Unlike the U.S. Census, which is conducted every 
decade, the New York survey is conducted annually. 


Los Angeles Unified’s School Experience Survey is especially relevant for 

California. 

mw Students are asked questions such as whether adults know their name, 
whether the school is clean, whether they feel safe while on campus, and 
whether what they are learning “takes a lot of thinking.” 


mw Parents are asked questions like whether they “feel welcome to participate 
at school,” whether they talk with teachers about their child’s schoolwork, 
and whether they feel their children are safe. 


@ School site personnel weigh in on issues such as whether they get the help 
they need to communicate with parents and whether they are “proud” of 
their school. 


The survey results are incorporated into a multicolored, multilingual, 
user-friendly report card that the district issues for each of its hundreds 


NOTES 
28 Education Code Section 52066. 
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of schools. The report card, printed in a colorful 11- by 22-inch format, 
prominently incorporates views of parents, students and staff, based on 
their survey responses. 


The report cards are easily accessed on the district’s interactive web 
page. The report card for John Marshall High School, for example, underscores 
the usefulness of the survey in giving a more three-dimensional view of the 
school. The school had an Academic Performance Index of 685, way below 
the proficiency goal of 800 set by the state. Yet parents, teachers and 
students express a high degree of satisfaction with the school. If outsiders 
did not have access to that information, they would have an entirely different 
perception of the school. 


One of the main challenges for the district has been getting a sufficient 
number of responses from parents so that survey results are statistically 
valid. 


In the elementary grades, children were given the surveys to take home, which 
has worked reasonably well. The main challenge has been getting parents of 
middle and high school students to respond. In the past, surveys were mailed 
to parents’ homes, but only about one in four parents responded. Instead of 
mailing the surveys, the district has boosted parent participation by sending 
them to middle and high schools, where school staff take responsibility for 
getting the surveys to parents, either by sending them home with students or 
by having parents fill out the surveys when they come to the school. 


Because of cost constraints, it may be difficult, and in many cases 
impossible, for many school districts in the state to conduct surveys like the 
one in Los Angeles. But it does present a model that districts could adapt to 
fit their own purposes. 


Other valuable surveys are the California School Climate Survey and the 
California Healthy Kids Survey, which the majority of California school 
districts receiving funds from the federal Safe and Drug Free Schools and 
Communities program were required to administer between 2004 and 2009. 
Eighty-five percent of California’s nearly 1,000 school districts participated 
in the surveys developed on behalf of the California Department of Education. 


Because districts are no longer required to administer the surveys, the 
number of participating districts has currently dropped by about one-third 
from its previous levels. But the majority of California districts still do 
participate. The research firm WestEd has now consolidated the two surveys, 
along with a parent survey, into a three-part survey instrument called the 
California School Climate, Health, and Learning Survey (CAL-SCHLS). The 
costs of administering the survey is relatively low because the California 
Department of Education underwrites it. 
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“Computerized assessments 
allow teachers, principals 
and parents to receive results 
in weeks, not months. Faster 
results mean that teachers 
can use the information... 
throughout the school year to 
differentiate instruction and 
better meet the unique needs 
of their students.” 


— SMARTER BALANCED ASSESSMENT 
CONSORTIUM 


NOTES 


29 Smarter Balanced Assessment Consortium 
website. 


ESSENTIAL PRINCIPLE #3: From a system based on tests that provide 

delayed feedback and are used primarily to rank schools and districts 

with those that give more immediate feedback in ways that help chil- 
dren learn and teachers teach more effectively. 

A major drawback of the California Standards Tests that the state has admin- 
istered for the past 16 years to students each spring is that they have provided lit- 
tle useful information to teachers, parents or students about individual students. 
@ Parents typically received their children’s scores in the mail during the sum- 

mer, without any explanation as to what they mean except whether the stu- 

dent is performing at a below basic, basic, proficient or advanced level. 

™ Parent and teachers received no breakdown as to the student’s strengths 
or weaknesses or in what areas he or she could have benefitted from addi- 
tional help. 

lM The tests have almost no meaning for students because they have no impact 
on their grades or whether they will be promoted to the next grade or gradu- 
ate from high school. 

@ Asa result, many students were not motivated to do well on the tests and may 
not have taken them seriously. This attitude, in turn, may have contributed to 
lower scores for individual students and the overall school and district. 

In contrast, a major thrust of the Smarter Balanced assessments that stu- 
dents will take for the first time in 2015-16 will be to provide information that 
will be useful to teachers and that will help students understand more clearly 
their strengths and weaknesses. 

Wherever feasible, students will be expected to take the new assessments 
using computers, providing rapid feedback to students and teachers. According 
to the Smarter Balanced Assessment Consortium: 

Computerized assessments allow teachers, principals and parents to 
receive results in weeks, not months. Faster results mean that teachers 
can use the information from optional interim assessments throughout 
the school year to differentiate instruction and better meet the unique 
needs of their students. 

The assessments will themselves have three components: a summative assess- 
ment, interim assessments and formative assessment practices and strategies. 

The summative assessment, as its name implies, is intended to measure the 
“sum” of what students have learned during the school year. As described by 
the Smarter Balanced Assessment Consortium, its purpose will be to provide 
“valid, reliable and fair measures of students’ progress toward, and attainment 
of, the knowledge and skills required to be career and college ready.” *” 

It will be administered during the last quarter of the school year and will 


consist of two parts: a “computer adaptive” component and “performance tasks.” 
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With computer adaptive tests, the computer program adjusts the level of 
difficulty of a question based on a student’s response to the previous one. As the 
Smarter Balanced Assessment Consortium has explained: 

By adapting to the student as the assessment is taking place, these 
assessments present an individually tailored set of questions to each 
student and can quickly identify which skills students have mastered. 
This approach represents a significant improvement over traditional 
paper-and-pencil assessments used in many states today, providing more 
accurate scores for all students across the full range of the achievement 
continuum.*° 

Performance tasks are those that require an active response from students, 
rather than simply asking them to answer a multiple-choice question. So, for 
example, on the Smarter Balanced assessments, a student may be asked to sup- 
ply an ending for a story, or to flesh out a paragraph on a topic such as why the 
school day should be longer. Smarter Balanced says that it is developing “auto- 
mated technologies” to score written responses, and that items that can’t be 
scored automatically will be evaluated by trained evaluators.”™ 

Teachers will also be able to administer optional interim assessments at 
intervals still to be established by local schools and districts. These are intended 
to provide teachers “with actionable information” about student progress during 
the year. The goal of these interim assessments will be to help teachers, students 
and parents understand whether students are on track, and help them identify 
their strengths and weaknesses during the course of the year. 

Finally, the new assessments will also encourage the use of optional forma- 
tive assessments*—activities and assignments that are integrated into daily 
classroom instruction. They give teachers feedback as to how their students are 
doing and help students understand their strengths and weaknesses, and what 
areas of the curriculum they need to work on most. 

If indeed the Smarter Balanced assessments include these three dimensions, 
they will represent a major advance on the once-yearly tests students currently 
take. Rather than simply providing information demonstrating whether a school 
is meeting federal or state mandates, educators will receive useful information 


to help drive a student’s academic progress. 
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“Most people are not 
actually motivated by 
rewards and punishments.... 
They’re motivated by 

doing a good job. 

That's especially true 

in education.” 


— LINDA DARLING-HAMMOND, 
PROFESSOR OF EDUCATION, 
STANFORD UNIVERSITY, AND CHAIR, 
CALIFORNIA COMMISSION ON 
TEACHER CREDENTIALING 
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ESSENTIAL PRINCIPLE #4: From a system that is primarily based on 
external rewards and punishments to one that builds in intrinsic in- 
centives to motivate change among students, teachers and schools. 

This may be the most challenging principle to incorporate into California's 
emerging testing and accountability system—to motivate both students and 
teachers to do well, independent of the external rewards and sanctions that have 
characterized the state’s accountability systems until now. 

The central notion—argued, among others, by Michael Fullan, the Canadian 
educator and expert on organizational change—is that the rewards of acquiring 
knowledge and mastering skills will motivate students to succeed. Similarly for 
teachers, a system that is designed to help them experience success with their 
students will in the long run yield more impressive results. 

These notions are diametrically opposed to the main approach of the No 
Child Left Behind law. As Heinrich Mintrop and Gail Sunderman noted in 
their paper for The Civil Rights Project at UCLA, Why High Stakes Accountabil- 
ity Sounds Good, but Doesn’t Work: 

‘The federal accountability system, made universal through the No Child 
Left Behind Act of 2002, is at its heart a quota and sanctions system. This 
system stipulates the progression of underperforming schools through a 
set of increasingly severe sanctions based on meeting performance quotas 
for specific demographic groups.* 

That approach to testing and accountability, argues Linda Darling- 
Hammond, the Stanford educator who is also chair of the California Commis- 
sion on Teacher Credentialing, will not yield the results the system is seeking. 
As she said in an interview with EdSource: 

Most people are not actually motivated by rewards and punishments, 
although those can produce various kinds of behaviors that try to drive 
people in a given direction. They're motivated by doing a good job. That’s 
especially true in education. When you prepare people well and support 
them in a collaborative organization for ongoing learning, then you're 
actually providing the grit for a high level of commitment and intrinsic 
motivation because people are more efficacious when they are well pre- 
pared, and if they feel they can succeed. That drives higher commitment 
and motivation to succeed. 

Ideally, the motivation to do well on tests would start with motivating stu- 
dents themselves. Yet, discussion of how to incorporate strategies to motivate 
students to do better has been mostly absent from the national reform agenda. 


Instead, the major focus has been on getting adults to help students do better. 


© CopyRiGHT 2014 BY EpSourcE, INc. 


EDSOURCE REPORT 


NOTES 


35 Usher, Alexander and Nancy Kober. Student 
Motivation: An Overlooked Piece of School Reform, 
Center on Education Policy, May 2012. 


36 Fullan, Michael. Choosing the Wrong Drivers for 
Whole System Reform, Centre for Strategic Educa- 
tion Seminar Series, Paper No. 204, May 2011. 


As a paper by the Center on Education Policy, Student Motivation: An Over- 
looked Piece of School Reform, pointed out: 
Motivation is a central part of a student’s education experience, but it 
has received scant attention amid an education reform agenda focused 
mainly on accountability, standards and tests, teacher quality, and 
school management. 
The paper defines two kinds of motivation: 

1. Extrinsic motivation “is the desire to do or achieve something not so 
much for the enjoyment of the activity itself, but because it will produce a 
certain result.” 

2. Intrinsic motivation is “the desire to do or achieve something because one 
truly wants to and takes pleasure in doing it.” 


> 


Some schools and districts have tried to stimulate students’ “extrinsic moti- 
vation” by offering external rewards, such as pizza coupons or even cash incen- 
tives in exchange for getting better grades, reading more books or doing well on 
tests. However, these incentives have produced limited results, according to the 
Center for Education Policy’s report: 

The most important conclusion is that reward programs that get the 

most results are ones that focus on mastery—demonstrating more 

understanding, skills and content knowledge—not on performance- 

based goals, such as having students reach a pre-defined goal (such as 

a score or a grade). 

For Michael Fullan, fostering intrinsic motivation among students is an 
essential ingredient in any effective accountability system. Fullan’s ideas are at 
the center of the reforms being implemented by the eight California school dis- 
tricts that received a No Child Left Behind waiver.’ 

In a critique of the Common Core standards and the Smarter Balanced 
assessments, he wrote: 

‘There is no way that these ambitious and admirable nationwide goals 
will be met with the strategies being used. No successful system in the 
world has ever led with these drivers. They cannot generate on a large 
scale the kind of intrinsic motivational energy that will be required to 
transform these massive systems. 

Extrinsic motivation has its place, but only in combination with intrinsic 
rewards, he argues. “People will do things longer and deeper if it is intrinsically 
meaningful to them; if it is extrinsically useful, they might do it for a while, but 


not for that long,” he asserted in an interview with EdSource. 


24 M@ REFORMING TESTING AND ACCOUNTABILITY HM MARCH 2014 © CopyRicGHT 2014 BY EpSourcE, INc. 


EDSOURCE REPORT 


Photo by Susan Frey 


NOTES 


37 Hunter, Phyllis S. Raising Students Who Want to 
Read, Scholastic Professional Paper. 


Even trying to convince students that they must do well in school—and on 
tests—so that they will be able to go to college has its limits. “The prospect of 
going to college will motivate some people, but it won't last until there is some 
deeper meaning,” he said. 

The larger challenge, said Fullan, is to tackle the problem of bored kids. “The 
goal is how to move from boredom to excitement about learning,” he said. 

Then there is the challenge of fostering intrinsic motivation among teach- 
ers. What they need, Fullan said, is to experience success with their students 
by using what he calls the social capital of a school—working collaboratively 
with other teachers, principal and staff, and making use of their experience and 
insights. He pointed to districts such as Sanger Unified in the Central Valley, 
Garden Grove Unified south of Anaheim, and Long Beach Unified as promot- 
ing such a culture within their schools. Said Fullan: 

Whether you do it via other teachers helping, or principals and literacy 
coaches, you want to give teachers new skills and new experiences that 
get successes like they have never had before; and as a result of that, their 
intrinsic motivation kicks in. 

Integrating the concepts of internal “drivers” of change and how best to moti- 
vate teachers into discussions of testing and accountability presents significant 
challenges. But looking afresh at strategies that educators have routinely used 
to motivate students,” and then figuring out how to introduce them into the 
broader arena of testing, could yield significant dividends, Fullan contended. 

When teachers get turned on, there is tremendous motivation and 
energy that goes into their work. It is something you can’t buy or can't 
get through ‘accountability.’ The goal is to turn that around, so it is 
worthwhile going to school for both teachers and students. 

Stanford’s Darling-Hammond agrees: 

If teachers get good information about how well they’re doing and how 
well their students are doing, and they're working in a learning orga- 
nization, they'll then also build on that and continue to improve their 
practice. I think we have a lot of evidence that that is how people in 
education are motivated to do well. 

The reforms initiated by the Local Control Funding Formula should move 
the state further in this direction. Instead of punitive sanctions, the new 
accountability system is designed to provide schools with the support they 
need to improve. An entirely new state agency, the California Collaborative for 
Educational Excellence, will be established to provide such support. Even in the 
case of “persistently failing schools,” the new agency must first have provided 


assistance and must indicate that state intervention is still necessary. 
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California has taken an 
important step forward 

by including the Next 
Generation Science 
Standards as one of the 
priorities that districts need 
to address. But the challenge 
remains to develop the 
curriculum, instructional 
materials and assessments 
to match the new standards. 
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ESSENTIAL PRINCIPLE #5: From a system that focuses disproportion- 
ately on math and English language arts to a more balanced one that 
incorporates other key subject areas, especially science. 

President Barack Obama has made science a central part of his education 
agenda. As he declared at the White House Science Fair in April 2013: 

We need to make this a priority to train an army of new teachers in these 
subject areas, and to make sure that all of us as a country are lifting up 
these subjects for the respect that they deserve.** 

California is taking a leadership role in being one of the early adopters of the 
Next Generation Science Standards. 

But that cannot obscure the fact that math and English language arts have 
dominated the curriculum and assessment landscape for the past decade and 
a half—to the detriment of other key subject areas including science, history/ 
social science, music, arts and physical education. 

Several reports during the past decade by the Center for Education Policy 
have found that, especially in underperforming schools, these key areas of the 
school curriculum have been trimmed as schools have been forced to focus 
intensively on math and English language arts. 

Because the most energy and resources have been invested in developing 
new science standards and work has begun on assessments, this report focuses 
on science. But that does not diminish the importance of elevating the status of 
other parts of the school curriculum as well. 

In terms of California’s emerging accountability system, science assess- 
ments have taken a back seat to those in math and English language arts. Under 
NCLB, science tests were required to be administered only once within each of 
K-s, 6-9 and 10-12 grade spans—and in California, they have been administered 
at grades s, 8 and 10. (There have also been end-of-course, single-subject stan- 
dards tests in high school biology, chemistry, earth science and physics.) 

Some progress is being made in the Common Core standards, which encour- 
age integrating science (and history/social science) standards and curriculum 
with those in math and English language arts.” 

But the Smarter Balanced Assessment Consortium, which is developing 
math and English language arts assessments, is not doing so explicitly for sci- 
ence. Unlike the massive federal investment in both the Smarter Balanced 
Assessment Consortium and the Partnership for Assessment for Readiness 
for College and Careers (PARCC) development of math and English language 
arts assessments for the Common Core, there has been no comparable federal 
investment in science assessments. 

Meanwhile, the state of science education remains a source of enormous 


concern to a broad range of educators and stakeholders. 
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Here is one influential critique from the National Academy of Engineering: 
The U.S. science curriculum is ‘a mile wide and an inch deep,’ covering 
a large number of topics in a scattershot, shallow fashion rather than 
systematically and in depth. These features are evident both in state and 
district standards and in textbooks (another important influence on the 
curriculum). The U.S. science curriculum stands in stark contrast to that 
of other nations, in particular high-achieving ones such as Japan.*° 

Similarly, the National Research Council’s Committee on a Conceptual 
Framework for New K-12 Science Education Standards found that: 

Currently, K-12 science education in the United States...is not organized 
systematically across multiple years of school, emphasizes discrete facts 
with a focus on breadth over depth, and does not provide students with 
engaging opportunities to experience how science is actually done. 

Two years ago, a National Academy of Sciences panel called on “state, 
national, and local policymakers ...to elevate science education in grades K-12 to 
the same level of importance as reading and mathematics.”* 

That has not yet occurred. 

California has taken an important step forward by including the Next Gen- 
eration Science Standards as one of the priorities that districts (including char- 
ter schools) need to address as part of the Local Control and Accountability 
Plan they must draw up by July 1, 2014. 

But the challenge remains to develop the curriculum, instructional materials 
and assessments to match the new standards. It will be several years before all 
these components are in place. 

In March 2014, the California Department of Education began to put together a 
team of about 25 key science education leaders in California and two liaisons from 
the State Board of Education. The team will develop an implementation plan that 
lays out what must done during the next three to four years to fully implement the 
science standards statewide. The implementation plan will be presented to the State 
Board of Education for approval at the July 2014 meeting. 

Developing quality assessments will be particularly challenging. A summary 
of two days of meetings of more than 200 experts brought together by the Edu- 
cation Testing Service and the College Board in September 2013 concluded on 
this salutary note: 

As made clear by many speakers, translating the Next Generation Sci- 
ence Standards to assessments presents numerous challenges, particu- 
larly for the design of consequential assessments if current constraints of 
testing time, cost and uses of data are to be adhered to.#* 

This spring, California students will take Common Core field tests devel- 
oped by the Smarter Balanced Consortium. But the tests will only be in math 


and English language arts. California will continue to administer the California 
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Standards Test in science, as required by federal law, in grades 5, 8 and 10. The 
existing single-subject, end-of-course high school CSTs in biology, chemistry, 
earth science and physics will be terminated. 

“We are in a holding pattern,” said Chris Roe, president of the California 
STEM Learning Network, referring to the continuing use of California’s old 
tests in some grades and the absence of new ones in any grade. “California is 
stuck between a rock and a hard place. Until there is something else in place, we 
don’t have much choice.” 

What is not clear is whether California will need to develop its own assess- 
ments or whether the state can be part of a consortium, such as the national 
Smarter Balanced Consortium for English and math. 

The federal government, which supported the effort to create two national 
consortia for math and English, has so far shown no interest in supporting the 
development of science assessments. California is informally discussing work- 
ing together with Western states, including Washington, Oregon and Nevada, 
according to Phil Lafontaine, director of the Professional Learning Support 
Division at the California Department of Education. 

“Tt doesn’t make sense to do it on our own,” he told EdSource, “but right now 
it’s just a lot of conversations—no formal agreements.” 

Lafontaine said it is unclear when new science assessments will be available. 
“It’s up in the air right now,” he said, adding that the department is eager for the 
new assessments to be implemented, but “we want there to be time to do it well.” 

Generating public support for the new science standards could play a role 
in both their quality and how quickly they are implemented. As the Califor- 
nia STEM Learning Network in concert with other organizations have noted: 
“It is essential that education, business, parent and community leaders become 
informed about the Next Generation Science Standards and make their desires 
known to policymakers.” 

An editorial in Science, co-authored by Jane Coffey from the Gordon and 
Betty Moore Foundation, and Bruce Alberts, the magazine’s editor-in-chief, 
says the involvement of scientists in the process is crucial. 

Urgently needed is a vigorous R&D agenda that pursues new methods 
of and approaches to assessment. This will be difficult but critically 
important long-term work. Outstanding scientists must be willing to 
work side-by-side with measurement specialists and science educators to 
develop methods for evaluating what is important to measure.** 

Trish Williams, a State Board of Education liaison to the science standards 
implementation team, said the board will work to ensure a “robust implemen- 
tation” of the science standards. “We are committed to California becoming a 


national leader in science education,” she said. 
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The new accountability 
system should encourage 
schools to challenge all 
students, no matter how 
accomplished, to continue 
stretching themselves 
academically, while not 
losing sight of the need to 
narrow gaps in achievement 
between different groups 
of students. 
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ESSENTIAL PRINCIPLE #6: From a system that has focused mainly on 
getting children to perform at a “proficient” level to one that meas- 
ures student growth from year to year, motivates all children to do 
better, and rewards students and schools for making progress. 

The major thrust of both the federal No Child Left Behind law and the Pub- 
lic Schools Accountability Act has been to focus on children who are not suc- 
ceeding, rather than rewarding those who are. On the whole, the same approach 
has applied to schools. 

There are no explicit incentives in NCLB for schools to improve the aca- 
demic performance of students after they have scored at a proficient level on 
state tests, or of students who were already at an “advanced” level. The same 
applies to the state system—after schools and their student subgroups reach the 
800 API target, they are no longer subject to the accountability provisions of 
the Public Schools Accountability Act. 

Clearly, most educators want all students under their care to perform at opti- 
mal levels. But it was not a goal built into the accountability systems. 

A new accountability system should hold all schools accountable for the 
academic success of all students regardless of socioeconomic status, race, com- 
munity type (urban, suburban, rural), and language and disability status. The 
system should encourage schools to challenge all students, no matter how 
accomplished, to continue stretching themselves academically, while not losing 
sight of the need to narrow gaps in achievement between different groups of 
students. 

One way to build in a goal that all children—and the schools that serve 
them—improve over time is to move to a system that measures actual growth of 
individual students, in place of the current system that measures how this year’s 
3rd grade class, for example, compares to last year’s 3rd grade class, regardless of 
whatever demographic or other differences might exist between the two classes. 

Fortunately, the new assessments that schools will begin administering this 
spring will move California in this direction. Paul Warren, in a PPIC paper titled 
California’s Changing K-12 Accountability System, points out that the Smarter Bal- 
anced assessments will use a “common scale” for the English and math tests in 
grades 3 through 8, which will allow California to measure the annual growth 
of individual students.* 

Because each student in California has a unique student identifier as part of 
a longitudinal tracking system (the California Longitudinal Pupil Achievement 
Data System or CALPADS), it is now possible for California to move from a 
“fixed target” model of accountability like the one embodied in No Child Left 
Behind to a “growth-based” one that can actually track student performance 


over time.*° 
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However, to make that happen the state must still figure out how to inte- 
grate student growth measures with student test results in its accountability 
system. In other words, what growth targets—the improvement that students 
must make from one year to the next—will schools and districts have to show 
to demonstrate they are succeeding? 

Several states, such as Illinois, have established accountability systems using 
growth targets based on scores of individual students.*” However, even in those 
states, growth is still only rewarded until students score at an advanced level 
or, as in Illinois’ case, they have “exceeded standards.” After students reach the 
“exceeds standards” level, the system makes no provision for rewarding them if 
they improve their scores. 

Setting growth targets in California’s emerging accountability system will 
require legislative action. State officials point out that in order to see what kinds 
of progress students make on the new Smarter Balanced assessments from one 
year to the next, it makes sense to wait until the assessments have been admin- 
istered for two years, and only then to recommend new growth targets. These 
targets should incorporate ways to motivate students already scoring at an 


advanced level to do even better. 
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California seems to be 
heading away from 

coming up with a single 
grade, index or score, 

and moving toward a 

more multidimensional 
accountability system. But it 
will take considerable work 
to figure out how to present 
information and data in 

a way that is accessible 
and useful for everyone. 


ESSENTIAL PRINCIPLE #7: From a complex testing and accountability 

system that is hard for ordinary Californians to understand to one 

that is more transparent, and presents a multidimensional portrait of 
how students and their schools are doing over time in straightforward 
language. 

California’s accountability system has been based largely on whether a school 
has met its target “growth measures” on the Academic Performance Index, a 
single number based mainly on how students do on California Standards Tests, 
and on whether a school is achieving adequate yearly progress or is in need of 
Program Improvement as defined by the federal No Child Left Behind law. 

At first glance, the API may seem like a simple score that ranks a school on a 
scale of 200 to 1,000. But the API is in fact a complicated and not very transpar- 
ent measure of school performance, as even a cursory glance at the annual Cali- 
fornia Department of Education manual explaining it will demonstrate. Few 
people who use the API know how it is reached, or what it really means. The 
same applies to knowing how a school is determined to have made “adequate 
yearly progress” under the No Child Left Behind law. The AYP determination 
is based on how multiple student subgroups have performed on any number of 
tests, how many students have reached proficiency levels, and how many stu- 
dents have taken the tests. 

It would be safe to say that most parents would not know a school can be 
deemed to have failed to make “adequate yearly progress” simply because less 
than 95 percent of students in any one of numerous subgroups failed to take the 
test. Even if just a handful of students in any subgroup don’t take the test, the 
entire school can be deemed not to have made adequate progress. 

Another issue that California has not addressed head-on is that there are 
multiple constituencies to which schools must be held accountable for their per- 
formance—taxpayers, legislators, parents, educators, employers and students— 
and that different audiences need information presented in different ways. 

m@ Employers and business leaders need to know if students are emerging pre- 
pared for college and careers. 

™@ Parents need to know if their children are being taught effectively and are 
expected to meet high academic standards. 

™ Taxpayers need to know their tax dollars are getting results. 

m@ Lawmakers need to know that schools are using state and federal funds in the 
ways they were intended, and are implementing state mandates and reforms 
in an effective fashion. 

m@ Educators in and outside the classroom need information that is action- 
able, timely and gives them guidance as to the changes they need to make to 


ensure that the children in their care achieve at optimal levels. 
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California education stakeholders have not engaged in an open discussion 
about how they want the quality of a school or district to be assessed. 

There is the appeal of generating a single score or index (such as the API), 
a grade such as the ones awarded to schools in Florida and New York City, or 
even a more multilayered percentage score such as the School Quality Improve- 
ment Index proposed by the CORE consortium of eight California districts. 
But there are also clearly advantages to schools or districts being evaluated on 
multiple measures, such as the eight “priority areas” prescribed by the Local 
Control Funding Formula legislation. 

The risk is that California will attempt to come up with a single measure that 
cannot possibly encapsulate the multidimensional aspects of a complex institu- 
tion like a school or district. At the same time, providing too much information 
on too many measures can confuse and obscure what is going on in a school 
or district. That is what has occurred with the School Accountability Report 
Card (or SARC) mandated by Proposition 98, the initiative approved by voters 
in 1988. 

The SARCs were intended to be useful documents that would allow the pub- 
lic to hold schools accountable for the expenditure of state funds—and for get- 
ting results. But over the years, they have turned into lengthy, often inscrutable 
documents weighted down by additional reporting requirements mandated by 
subsequent legislation. In many cases, they are simply public relations docu- 
ments intended to put schools in the best possible light. 

California seems to be heading away from coming up with a single grade, 
index or score, and moving toward a more multidimensional accountability sys- 
tem. But it will take considerable work to figure out how to present information 
and data in a way that is accessible and useful for everyone. 

It would be worthwhile to look at how other states present outcomes on 
multiple measures—such as Colorado’s SchoolView, New York State’s Report 
Cards, and New York City’s Progress Report Overview. These could be used to 
inform a new and much more simplified approach to presenting information in 
California. The School Quality Snapshot produced by the California Depart- 
ment of Education and the School Report Cards produced by Los Angeles Uni- 
fied are both good models to build on. 

California would also benefit from engaging stellar graphic designers and 
data visualization experts to come up with new ways to provide information 


that present meaningful and understandable portraits of a school’s quality. 
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If technology allows students 
to learn at their own 
pace—and advance more 
quickly—the “one size 

fits all” standardized 

tests should become far 

less central to the 
accountability system. 


NOTES 


48 “Fewer, better tests can boost achievement,” 
Education Week, Oct. 7, 2013. 


ESSENTIAL PRINCIPLE #8: From an assessment system that uses tech- 
nology mainly to report results to schools and the public to one that 
uses technology in innovative ways to provide more immediate feed- 
back to teachers and students, and that tracks students’ progress 
through the 12th grade and into college and the workplace. 

Because of California’s leadership in the high-tech world, and the creative 
energies and industry of Silicon Valley in particular, the state’s public schools 
should be the beneficiary of technological advances that could be applied to 
new assessment and accountability systems. 

Clearly, the goal of the computer-adaptive Smarter Balanced assessments— 
to allow students to move to easier or more demanding questions depending on 
how they respond to each question—would represent a major advance in the 
application of technology to assessments. 

What remains unclear is how many schools will have both the broadband 
capacity and the computer hardware to administer the Smarter Balanced assess- 
ments in their online versions, and how many will have to administer them in 
more traditional pencil-and-paper formats. 

Beyond administration of the Smarter Balanced assessments, successful 
applications of technology could be a key to transforming our current testing 
and accountability system as we know it. This transformation could make it 
possible to cut back sharply on testing simply for accountability purposes— 
the big annual “summative” tests students are now required to take—because 
there will be a continual stream of information about student learning available 
throughout the year. 

Some distinguished education leaders question the necessity of having mil- 
lions of students take tests for accountability purposes every year. In a recent Edu- 
cation Week article, Marc Tucker, president of the National Center on Education 
and the Economy, Linda Darling-Hammond, chair of the California Commission 
on Teacher Credentialing, and John Jackson, president of the Schott Foundation 
for Public Education, argue that standardized tests should be administered to just 
one grade at the elementary, middle and high school levels.** 

In place of annual tests, they argue, schools could use “higher quality assess- 
ments that encourage more productive teaching, while reducing the testing bur- 
den on students and teachers.” 

‘The countries that outperform the United States on international exams 
spend more than we do to measure and encourage these skills with essay 
tests and teacher-scored projects. And they can afford to do this because 
they test much less frequently than we do, typically only two or three 


times during a student’s entire school career. 
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NOTES 


49 Comments at Policy Analysis for Public Educa- 
tion Seminar, Sacramento, Oct. 2, 2013. 


Technology should make it possible to reduce the need for annual tests because 
teachers and students who are working online are able to get far more rapid—even 
instant—feedback on what they are doing. Already, teachers in many classrooms 
are giving students feedback as they are writing or completing assignments using 
tools like Google docs. Students are doing quizzes online and getting instant feed- 
back and suggestions for what they can do to improve their performance. Using 
tools like PowerSchool, students and their parents can get access to their scores 
and grades on tests and quizzes shortly after they take them. 

As more instruction gets delivered online, online programs will present 
classroom curriculum lessons that marry learning and assessment, with stu- 
dents and teachers getting ongoing real-time feedback. 

In addition, technology will also allow schools to move toward what educa- 
tors call a competency-based system, in which students advance when they have 
demonstrated competence or mastery in the knowledge expected by state and 
local standards. In other words, they can take tests and assessments throughout 
the school year to demonstrate that they have the necessary knowledge rather 
than having to wait to have their abilities tested at a fixed time every year, typi- 
cally in the spring. 

As the California Department of Education’s Deb Sigman has argued: 

We want assessments to be a way to promote teaching and learning. We 
want them to be an extension of teaching, not as an add in and drop in 
at some point during the year when students stop what they are doing 
and take an assessment that doesn’t look like what they have been doing 
in the classroom. We want assessments to look like what they have been 
doing all year round.* 

As a result, technology must necessarily be a core feature of any reformed 
accountability system, with potentially far-reaching impacts on the accountabil- 
ity system itself. If technology allows students to learn at their own pace—and 
advance more quickly—the “one size fits all” standardized tests administered 
annually in specific grades should become far less central to the accountability 
system than they have been in the past. 

At the same time, the capacity of schools and districts to use technology 
effectively will vary enormously. The state must ensure that school districts 
without the resources to take advantage of the opportunities presented by tech- 
nology in general, and educational technology in particular, are not left behind 


when it comes to effecting these transformations. 
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“Education is not the filling 
of a pail, but the lighting 
of a fire.” 


- WILLIAM BUTLER YEATS, QUOTED 
BY GOV. JERRY BROWN, 2013 STATE 
OF THE STATE SPEECH 


CONCLUSION 


What should be clear from this report is that the there are many strands and lev- 
els in the path toward establishing a new testing and accountability system— 
especially one that incorporates multiple measures and charts new territory by 
placing a priority on local involvement in evaluating school outcomes. 

One challenge is the sheer scale on which it must be done—in a state with 
nearly 1,000 districts, 10,000 schools and more than 6 million public school stu- 
dents—and the massive amount of data that represents. 

Another challenge is how to balance the development of a uniform testing 
and accountability system against the diversity of the state’s schools, geography 
and population. 

Yet another is how to move from a single score—the Academic Performance 
Index, which has been at the heart of the state’s accountability system—to one 
that assesses students on “deeper learning” skills that go beyond what can be 
tested by multiple-choice questions. What’s more, California must still develop 
new assessments in core subject areas such as science, history and social sci- 
ence, and the arts. 

To ensure success, there needs to be clear oversight of the process, and a way 
to manage the competing interests and complexities that are at play. 

It will also be important to ensure that the essential principles outlined in 
this report are incorporated into the new system, underscoring the need for 
ongoing oversight. 

If changes are required—including legislative ones—these need to be made 
early on in the process. For example, it may make sense to revisit Senate Bill 
1458, which mandates reforms in the Academic Performance Index. The bill 
was approved before the passage of legislation (Assembly Bill 97) reforming 
the state’s school finance system. As a consequence, the Academic Performance 
Index is only one of many measures that will be part of the state’s emerging 
accountability system. 

Now that parents, community representatives, teachers and administrators 
are required to be involved in recommending how state funds must be spent, 
a way for these local stakeholders to be included in reviewing the success or 
otherwise of the emerging accountability system must also be established. 

Their feedback will be crucial in helping to assess whether the new system 
ends up providing the information needed to contribute to better student out- 
comes, which ultimately is the goal of any testing and accountability system. 

It is also important to remember that the larger function of public schools 
should not be to simply prepare young people to do well on tests, which has been 
the major focus of accountability systems during the past decade and a half. 


It should be to let their talents—whatever they may be—flourish. Gov. Brown 
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has also emphasized the school’s role in “character formation’—an aspect of 
the educational experience that can’t be measured on tests. While standards are 
“fundamentally important,” he told the State Board of Education shortly after 
he took office in January 2011, “I really value creativity, innovation, the unusual, 
the alive, the vital.” 

California's emerging testing and accountability system must be aligned 
with these broader goals of schooling. But what is clear is that after more than 


a decade of implementing top-down assessments that have failed to deliver 


on their promises, California now has the opportunity to take the lead in the 


Shain rs Fey Ration in coming up with a system that not only fairly and accurately measures 
student and school performance, but also helps contribute to better teaching 
and learning. 

In order to make this happen, we recommend the following steps. 

m@ Publish a clear timeline: The California Department of Education in con- 
cert with the State Board of Education should publish a clear timeline for all 
key milestones, and distribute it widely to policymakers, parent and commu- 
nity organizations, school board members, and other education stakeholders 
to ensure that all major stakeholders are informed about the complex process 
for reforming the testing and accountability system. 

m@ Establish broader oversight: The State Board of Education should consider 
appointing an individual or group to oversee and coordinate the process and 
ensure that the necessary steps occur in a timely fashion, while still meet- 
ing the needs of students, schools and the state in general. Without overall 
coordination, there is a danger that the various elements of the new testing 
and accountability system will evolve in an uncoordinated fashion—and in 
potentially conflicting ways. 

m@ Promote dialog on the role of testing: The State Board of Education 
should initiate a statewide dialog about the role of testing in the state’s 
emerging accountability system and the best way to evaluate a school’s per- 
formance. This will be especially important as California moves away from 
its reliance on the Academic Performance Index as the primary way to evalu- 
ate school performance to one incorporating multiple measures as spelled 
out in school finance legislation and the Local Control and Accountability 
Plan that it mandates. 

m Engage in a deliberative process: Policymakers must move deliberately 
to put into place a new testing and accounting system—without doing so in 
an overly hasty fashion that yields unintended consequences. Because key 
pieces of California’s accountability system need only be in place in 2015-16, 
policymakers have time to be discerning in crafting a coordinated strategy 
that builds on the best approaches being developed within California and in 


other states. 
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m@ Develop new science assessments: Development of new science assess- 
ments is lagging behind those developed for English language arts and math. 
Educators concerned about developing meaningful assessments in science 
should be encouraged to participate in stakeholder discussions that will be 
launched in the next several months. Key stakeholders in business, academia 
and public policy have the opportunity to play a greater leadership role at a 
crucial time in not only ensuring that frameworks, curricula and assessments 
tied to the Next Generation Science Standards are developed in a timely way, 


but also in elevating the status of science within the school curriculum. 


m@ Conduct an ongoing review: After the key parts of the system are in place, 


Photo by Susan Frey 
a mechanism for continual review of the reforms must be instituted to ensure 
that they are meeting the needs of educators. The review mechanism must 
also provide educators with meaningful information about their students 
that contributes to greater academic success. Also, parents and other com- 
munity stakeholders should have a way to be involved in the statewide review 
process, paralleling their involvement at a local level. Ongoing review and 
About EdSource oversight will be especially critical because of the system’s reliance on tech- 
EdSource is an independent, impartial, nology, which will inevitably undergo major changes as the system is being 
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any cD implemented. 
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nians on key education challenges and to @ Provide adequate resources: While students and schools must be held 
PUSHIN SIGESSES WEE OTE SEN! accountable for achieving results, the state must provide the resources neces- 
success. 


sary for them to do so. The provision of the $1.25 billion to school districts 
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the California Collaborative for Educational Excellence that were intended 


California schools in general remain underfunded compared to most other 
states. There needs to be a mechanism to monitor the resources and fund- 
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University. presents enormous challenges and pitfalls, but it is one that also is essential for 


ensure that districts have the resources to successfully implement the new 
Reforming California’s assessment and accountability system is a task that 


the state to undertake as it prepares its young people for a future that neither 


their teachers, nor they, can fully anticipate—but must embrace. []f] 
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